A conjugate neural representation of visual objects in three dimensions

An arbitrary movement of a rigid object in 3-D space can always be decomposed instantaneously into a translation plus a rotation. How should the neural representation of a static view of a 3-D object be updated according to its instantaneous motion? Assuming that a motion-sensitive neuron detects the rate of change of an arbitrary function of the static view of an object, we predict that the generic response properties of such a neuron can always be specified by a preferred translation direction and a preferred rotation axis in 3-D space, with cosine directional or axial tuning and linear firing-rate modulation by speed or angular speed. This theoretical framework includes known properties of some motion-sensitive cells as special cases. From the activity of a population of these neurons, a feedforward network can readily extract the instantaneous rotation axis, the angular speed, and the translation velocity, and thereby completely determine the motion of the object. We propose that neurons tuned to 3-D motion (same view, different possible rotation axes) and neurons tuned to static views (same rotation axis, different possible views) can form a conjugate representation of a 3-D object. In this way, the internal neural representation can contain both static and dynamic information and effectively mimic how the static view of an object changes during arbitrary movement.